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Address Range 
502 


Home 


Mode 
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Client 3 


PTP 
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Client 3 


BC 
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Configure Address 
Network 
300 



Address 
Network receives 
Client Request 
302 





Yes 




f 


Determine mode 


304 




r 



Broadcast Request 
308 



Yes 




Convey Request 
Point-to-Point 
310 
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Initialize Address 
Network 
400 




Yes 



Suspend Clients 
404 f 




f 



Complete 
outstanding 
transactions 
406 
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r 


Change Mode 
408 
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Packet 
Type 


Full Name 


Address Space 


Description 


Cacheable 


I/O 


RTS 


ReadToShare 


Y 




Requests read-only copy of cache line 


RTO | 


ReadToOwn 


Y 




Requests writable copy of cache line 


RTWB 


ReadToWriteBack 


Y 




Requests to receive writable copy of cache line 
and send cache line to memory 


RS 


ReadStream 


Y 




Request read-once copy of cache line 


WS 


WriteStream 


Y 




Request to write entire cache line and send to 
memory 


WB 


WriteBack 


Y 




Request to send cache line from owning device 
to memory, device does not keep copy 


WBS 


WriteBackShared 


Y 




Request to send cache line from owning device 
to memory, device keeps read-only copy 


RIO 


ReadIO 




Y 


Request to read 10 locations 


WIO 


WritelO 




Y 


Request to write 10 locations 


INT 


Interaipt 






Sends an interrupt, target is specified by 
address 



Fig. 7 



Transaction Type 


Initiator Receives 


Initiator Sends 


RTS 


DATA 




RTO 


DATA 




RTWB 


DATA & PRN 


DATA 


RS 


DATA 




WS 


ACK & PRN 


DATA 


WB 


PRN 


DATA or NACK 


WBS 


PRN 


DATA or NACK 
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Transaction Type 


Initiator Receives 


Initiator Sends 


RIO 


DATA 




WIO 


PRN 


DATA 


INT 


PRN or NACK 


DATA or Nothing 



Fig. 9 



Access Rights Symbol 


Access Rights Name 


Description 


Data Present? 


W 


Write 


Read and Write 


Yes 


A 


All-Write 


Write-only, must write entire cache line 


Yes (or ACK) 


R 


Read 


Read-only 


Yes 


T 


Transient-Read 


Read-only, read can be reordered 


Yes 


1 


Invalid 


No access rights 


Yes or No 



Fig. 10A 



Ownership Status Symbol 


Ownership Status Name 


Description 


Data Present? 


0 


Owner 


Owns cache line 


Yes or No 


N 


Not Owner 


Does not own cache line 


Yes or No 



Fig. 10B 



Access Right 


Ownership Status 


W 


O 


R 


O 


I 


0 ! 


W 


N 


A 


N 


R 
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N 
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Transaction Type 


New Owner 


RTS 


Previous Owner 


RTO 


Initiator 


RTWB 


Memory 


RS 


Previous Owner 


ws 


Memory 


WB 


Memory (or Previous Owner) 


WBS 


Memory (or Previous Owner) 



Fig. 11 



RTS 




Fig. 12A 



RTO 
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Fig. 12C Fig. 12D 



WB 




Fig. 12F 
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RTO 



Wait 




Fig. 13A 




Fig. 13B 




Fig. 13C 



RTO 




Fig. 13D 
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Data Out Data In Address In Address Out 



i 


i 




i 


i 




Data and Address Queues 
720 



Promise 
Array 
904 



Transitory 

State 
Controller 
902 



Interface Controller 900 



Cache 
710 



State Info 
712 



Processing Unit 
702 
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StateDescription 


WOfh 


Waiting for WAIT and ACK for local WS, has wnte access and ownership 


WO 


Stable state with wnte access and ownership 


ROfn 


111 £ _ 1 A 1 A 1 - 1 A S~\ 1 X * ■ 1 1 A 1 A 1 • < ■ » 

Waiting for WAIT and ACK for local WS, has read access and ownership 


ROe 


Waiting for DATA for local RTO, has read access and ownership 


ROde 


Waiting for INV and DATA for local RTO, has read access and ownership 


ROd 


Waiting for INV for local RTO, has read access and ownership 


ROce 


Waiting for WAIT and DATA for local RTO, has read access and ownership 


RO 


Stable state with read access and ownership 


lOe 


Waiting for DATA for local RTO, has no access rights, has ownership 


lOde 


Waiting for INV and DATA for local RTO, has no access rights, has ownership 


lOd 


Waiting for INV for local RTO, has no access lights, has ownership 


WNj 


Able to send DATA/NACK for local WB/WBS, has write access and no ownership 


WNi 


Able to send DATA for local RTWB, write access, no ownership 


WNh 


Waiting for ACK for local WS, has write access, no ownership 


WNe 


Waiting for WAIT for local RTO, has write access, no ownership 


WN 


Stable state with write access, no ownership (caused by foreign transaction that took ownership, but 
for which no copyback has yet been performed) 


ANi 


Able to send DATA for local WS after performing write to entire cache line, no ownership 


RNj 


Able to send DATA/NACK for local WB/WBS, has read access and no ownership 


RNh i 


Waiting for ACK for local WS, has read access, no ownership 


RNg 


Waiting for INV for local WS, has read access, no ownership 


RNe 


Waiting for DATA for local RTO, has read access, no ownership 


RNde 


Waiting for INV and DATA for local RTO, has read access, no ownership 


RNd 


Waiting for INV for local RTO, has read access, no ownership 


RNce 


Waiting for WAIT and DATA for local RTO, has read access, no ownership 


RNcd 


Waiting for WAIT and INV for local RTO, has read access, no ownership 


RN 


Stable state with read access, no ownership 


TN 


Performing read(s), which may be reordered, for local RTS or RS, no ownership 


INk 


Waiting for RTO, RTWB, WS, WB or WBS after receiving an ERR or ERRL. No access rights, no 
ownership 


INj 


Able to send DATA/NACK for local WB/WBS, has no access and no ownership 


■ ML 

iNn 


Waiting for ACK for local WS or for DATA for local RTWB, has no access nghts, no ownership 


INg 


Waiting for INV for local WS or for INV for local RTWB, has no access rights, no ownership 


INe 


Waiting for DATA for local RTO, has no access rights, no ownership 


INde 


Waiting for INV and DATA for local RTO, has no access rights, no ownership 


INd 


Waiting for INV for local RTO, has no access rights, no ownership 


INce 


Waiting for WAIT and DATA for local RTO, has no access rights, no ownership 


INcd 


Waiting for WAIT and INV for local RTO, has no access rights, no ownership 


INa 


Waiting for DATA for local RTS, DATA may grant read access, has no access rights, no ownership 


IN 


Stable state with no access rights, no ownership 



Fig. 15A 
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Action 
Code 


Meaning 


Comments 


la 


Commit to send an ACK packet as a 
copyback by appending an entry for the 
received foreign packet in a copyback list. Set 
copy tag to 1. 


ACK packet may be sent from any state that allows 
copyback packets to be sent. It must be sent within a 
finite time of first entering such a state, regardless of 
what other oackets have been received 


cl 


Commit to send DATA and/or ACK packets 
for all outstanding copybacks for this cache 
line. Next, set copy tag to W. Then, perform 
state transition based on current state & 
local packet being received 


If sending copybacks changes the state from a state 
X to a state Y, the local packet being received will be 
received in state Y (and as a result, the entry for state 
Y in the table must be consulted to determine the 
state transition caused by receiving the local packet). 


Id 


Commit to send DATA packet for local RTWB, 
WS, WB, or WBS transaction. DATA packet is 
sent in response to receiving a PRN packet 
for this transaction. 


DATA packet may not be sent until a PRN packet is 
received. It must be sent within finite time of receiving 
the PRN packet & having entered a state that permits 
the packet to be sent, regardless of what other 
packets have been received. 


el 


Clear outstanding copyback commitments for 
this line by removing them from the copyback 
list. Do not send DATA or ACK packets for 
entries that were on the copyback list. Next, 
perform state transition based on current state 
& local packet being received. 


The action code is used in response to receiving an 
ERR or ERRL packet. If an ERR packet was 
received in place of a PR or PRACK packet, or if an 
ERRL packet was received in place of a DATAP i 
packet, a DATA packet may be sent to the error 
device. 


/i 


Commit to send a DATA packet as a 
copyback by appending an entry for the 
received foreign packet in the copyback list. 
Set coov taa to I 


DATA packet may be sent from any state that allows 
copyback packets to be sent. It must be sent within 
finite time of first entering such a state, regardless of 

what nthpr narkpfQ hav/Ck hopn roroiworl 
wiiaiuuici (JaLrftClo Have Uccll icucIVCU. 




bet wnte tag to 1. 




In 


Commit to send NACK packet for local WB or 
WSB transaction & set write tag to W. NACK 
packet is sent in response to receiving a PRN 
packet for this transaction. 


NACK packet may be sent at any time after receiving 
the PRN packet. It must be sent within finite time of 
receiving the PRN packet, regardless of what other 
nackets have bppn rerpivpd 


It 


Commit to send DATA packet as a copyback 
by appending an entry for the received foreign 
packet in the copyback list. If copy tag is W, 
set copy tag to R. 


DATA packet may be sent from any state that allows 
copyback packets to be sent. It must be sent within 
finite time of first entering such a state, regardless of 
what other packets have been received. 


Is 


Spt writp tan to R 

vJCl V¥l IIG luU l\J Fx. 




/w 


Commit to send DATA packet as a copyback 
by appending an entry for the received foreign 
packet in the copyback list. 


DATA packet may be sent from any state that allows 
copyback packets to be sent. It must be sent within 
finite time of first entering such a state, regardless of 
what other packets have been received. 


/y 


If copy tag is R, set copy to 1. 


Used to record invalidating transactions while a 
copyback for a foreign memory remap is pending. 


lz 


If write tag is R, set write to 1. 


Used to record invalidating transactions while a local 
WBS transaction is pending. 



Fig. 15B 
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RTO (1) 




RTO (2) 



Fig. 16 



RTS 
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100 



140A 



Address Network 
150A 



n 



Processing 
Subsystem 
142AA 



Processing 
Subsystem 
142BA 



Memory 
Subsystem 
144AA 



Memory 
Subsystem 
144BA 



I/O 
Subsystem 
146A 



Interface 
148 A 



Data Network 
152A 



140B 



Address Network 
150B 



Processing 
Subsystem 
142AB 



Processing 
Subsystem 
142BB 



Memory 
Subsystem 
144AB 



Memory 
Subsystem 
144BB 



I/O 
Subsystem 
146B 



Interface 
148B 



Data Network 
152B 



n 



Address Network 
150C 



140C 



Processing 
Subsystem 
142AC 



Processing 
Subsystem 
142BC 



Memory 
Subsystem 
144AC 



Memory 
Subsystem 
144BC 



I/O 
Subsystem 
146C 



Interface 
148C 



Data Network 
152C 
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gTag 


Description 


gM 


The maximum access right within the node is Write Access 


gs 


The maximum access right within the node is Read Access. 
No client device in the node can have Write Access. 


gi 


The maximum access right within the node is Invalid Access. 
No client device in the node can have Read or Write Access. 



Fig. 21 
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Packet 
Tvoe 


Full Name 

■ mm ■ ■ mm %m III \+ 


Address Space 


Description 


Cacheable 


I/O 


PRTS 


Proxy RTS 


Y 




Request from an interface in a gS or gi node in 

rp<snnn^p to a RTS rpmip^t frnm annthpr nnHp 

1 COjJVJI IOC LVJ a 1 A 1 O 1 CLjUCOl II VJI J l al IUII Id 1 IUUC 


PRTSM 


Proxy RTS Modified 


Y 




Request from an interface in a gM node in 

icoJJUHoc IU d f \ 1 O I cLjUcol II Ull 1 dflUUIt;! llUUc 


PRTOM 


Proxy RTO Modified 


Y 




Request from an interface in a gM node in 
response to a RTO request from another node 


PRTO 


Proxy ReadToOwn 


Y ; 




Request from an interface in response to a 
RTO request from another node 


PU 


Proxy Upgrade 


Y 




Request from an interface asking memory to 
supply data for an outstanding RTO 


PDU 


ProxyDataUpgrade 


Y 




Request from an interface asking memory to 
update gTag to gM; interface supplies data for 
an outstanding RTO 


PRSM 


Proxy ReadStream 

IVIUUIIIcU 


Y 




Request from an interface in a gM node in 
response 10 rco request irom anoiner nooe 


PIM 


Proxylnvalidate 

iviuunicu 


Y 




Request from an interface in a gM node to 
invalidate aata in cacnes ana memory 


PI 


Proxylnvalidate 


Y 




Request from an interface in a gS or gl node to 
invalidate data in caches and memory 


PMR 


ProxyMemoryRead 


Y 




Request from an interface to memory to read 
coherency state(s) and data or meta-data 


PMW 


ProxyMemoryWrite 


Y 




Request from an interface to memory to write 
coherency state(s) and data or meta-data 


Fig. 22 


Packet 
Type 


Full Name 


Description 


DATAM 


Data-Meta 


Data packet containing data and coherence state information 


DATAN 


Data-NoPull 


Data packet sent in response to PRTSM indicating no PRN will be 
coming 


REP 


Report 


Report from memory to an interface indicating a transaction to be 
handled by the interface 



Fig. 23 
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Subtransaction 


Preexisting 

Global Access State 


New Global Access State 


PRTSM 


Modified 


Shared 


PRTOM 


Modified 


Invalid 


PRTO 


Shared, Invalid 


Invalid 


PU 


Shared 


Modified 


PDU 


Shared, Invalid, Modified 


Modified 


PRSM 


Modified 


Unchanged 


PIM 


Modified 


Invalid 


PI 


Shared, Invalid 


Invalid 


PMR 


Shared, Invalid, Modified 


Unchanged 


PMW 


Shared, Invalid, Modified 


Equal to new global access state 
specified in DATAM packet 



Fig. 24 



REPLACEMENT SHEET 

28/43 




Fig. 25 
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Fig. 27 
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Home Node 140H 




Slave Node 140S 



Fig. 28 
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r 



830 



r 



840 



Data Out 



Data In 



Address In 



Address Out 
i 












Request Agent(s) 
802 






Home Agent(s) 
804 

Global Information 
850 






Slave Agent(s) 
806 











Interface 148 



Communications In/Out 
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Fig. 31 
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Requesting Node 140R 
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Home Node 140H 




Requesting Node 140R 
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Fig. 34 
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Response 
Info 


Coherence 
State 


Home Memory Subsystem Action 


No 


gM 


BC Mode — Allow OWnina dPX/ir<= tr» rpQnnnrl 

PTP Mode- Forward response to owning 
device 


No 


gs 


Send REP packet to interface if write access 
requested 


No 


gi 


Send REP packet to interface 


Yes 


gM 


Respond with copy of the requested 
coherency unit 



Fig. 35 



Response 
Info 


Memory Subsystem's Action 


mN 


Does not respond with copy of 
coherency unit because a cache within 
the node owns the coherency unit 


mR 


Does respond because memory is the 
owner 


mS 


Does respond to requests for shared 
access because memory has shared 
copy and no active device has 
ownership; does not respond to 
requests for write access 


ml 


Does not respond because memory's 
copy is invalid 



Fig. 36 
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Slave Node 140S 




Requesting/Home Node 140H 
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: I 

Send PRTOM, PRTSM, PIM, orPRSM 
specifying the coherency unit 
500 



Respond = false 
502 



Current record = oldest record in OTQ 




specifying the coherency unit 




504 






. 


r 




Respond = 

False: Receive data packet 
from active device in 
response to packet sent at 
500. Send received data in 
coherency message 

True: Send data from buffer 
in coherency message 

512 



Current record = 

RTO: Send data packet using data from 
buffer if respond = true & set respond = 
false 

RS, RTS: Send data packet using data 
from buffer if respond = true 

WB, WBS: Send PRN packet. Set 
respond = true if responsive DATA 
packet received & store received data in 
buffer 

WS, RTWB: If respond = false, send 
PRN & set respond = true & store 
received data in buffer. If respond = true, 
send PRACK or DATAP 



508 



Current record = next oldest record in 
OTQ specifying the coherency unit 
510 







Fig. 37 A 
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Packet 
Type 


Full Name 


Description 


RWB 


Remote WB 


Request sent from an active device in a multi-node system 
to an interface in order to initiate a WB transaction 


RWBS 


Remote WBS 


Request sent from an active device in a multi-node system 
to an interface in order to initiate a WBS transaction 


RWS 


Remote WS 


Request sent from an active device in a multi-node system 
to an interface in order to initiate a WS transaction 



Fig. 39 
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/ lnterface\y 




^ H 148R f 



Data/ 

Acknowledgment 



Home 
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/ Interface^ 




^ 1 148H J 


PMW/ 






PRH/ / 
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Requesting Node 140R 




Fig. 41 



